{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Dataflowr](https://raw.githubusercontent.com/dataflowr/website/master/_assets/dataflowr_logo.png)](https://dataflowr.github.io/website/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 2b: Playing with pytorch: linear regression\n",
    "\n",
    "[Video timestamp](https://youtu.be/Z6H3zakmn6E?t=960)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zOgNQwiv8We1"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "smvq6dbY8We4"
   },
   "outputs": [],
   "source": [
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XsESRui28Wgy"
   },
   "source": [
    "## Warm-up: Linear regression with numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FvAi4z478Wg0"
   },
   "source": [
    "Our model is:\n",
    "$$\n",
    "y_t = 2x^1_t-3x^2_t+1, \\quad t\\in\\{1,\\dots,30\\}\n",
    "$$\n",
    "\n",
    "Our task is given the 'observations' $(x_t,y_t)_{t\\in\\{1,\\dots,30\\}}$ to recover the weights $w^1=2, w^2=-3$ and the bias $b = 1$.\n",
    "\n",
    "In order to do so, we will solve the following optimization problem:\n",
    "$$\n",
    "\\underset{w^1,w^2,b}{\\operatorname{argmin}} \\sum_{t=1}^{30} \\left(w^1x^1_t+w^2x^2_t+b-y_t\\right)^2\n",
    "$$\n",
    "\n",
    "[Video timestamp](https://youtu.be/Z6H3zakmn6E?t=1080)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_hSr09Qa8Wg1"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from numpy.random import random\n",
    "# generate random input data\n",
    "x = random((30,2))\n",
    "\n",
    "# generate labels corresponding to input data x\n",
    "y = np.dot(x, [2., -3.]) + 1.\n",
    "w_source = np.array([2., -3.])\n",
    "b_source  = np.array([1.])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Vclk_vje8Wg2"
   },
   "outputs": [],
   "source": [
    "x[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "S5F3fcVN8Wg6"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "\n",
    "def plot_figs(fig_num, elev, azim, x, y, weights, bias):\n",
    "    fig = plt.figure(fig_num, figsize=(4, 3))\n",
    "    plt.clf()\n",
    "    ax = Axes3D(fig, elev=elev, azim=azim)\n",
    "    ax.scatter(x[:, 0], x[:, 1], y)\n",
    "    ax.plot_surface(np.array([[0, 0], [1, 1]]),\n",
    "                    np.array([[0, 1], [0, 1]]),\n",
    "                    (np.dot(np.array([[0, 0, 1, 1],\n",
    "                                          [0, 1, 0, 1]]).T, weights) + bias).reshape((2, 2)),\n",
    "                    alpha=.5)\n",
    "    ax.set_xlabel('x_1')\n",
    "    ax.set_ylabel('x_2')\n",
    "    ax.set_zlabel('y')\n",
    "    \n",
    "def plot_views(x, y, w, b):\n",
    "    #Generate the different figures from different views\n",
    "    elev = 43.5\n",
    "    azim = -110\n",
    "    plot_figs(1, elev, azim, x, y, w, b[0])\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8l7XSyJM8Wg8"
   },
   "outputs": [],
   "source": [
    "plot_views(x, y, w_source, b_source)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FCZAMgQn8Wg9"
   },
   "source": [
    "In vector form, we define:\n",
    "$$\n",
    "\\hat{y}_t = {\\bf w}^T{\\bf x}_t+b\n",
    "$$\n",
    "and we want to minimize the loss given by:\n",
    "$$\n",
    "loss = \\sum_t\\underbrace{\\left(\\hat{y}_t-y_t \\right)^2}_{loss_t}.\n",
    "$$\n",
    "\n",
    "To minimize the loss we first compute the gradient of each $loss_t$:\n",
    "\\begin{eqnarray*}\n",
    "\\frac{\\partial{loss_t}}{\\partial w^1} &=& 2x^1_t\\left({\\bf w}^T{\\bf x}_t+b-y_t \\right)\\\\\n",
    "\\frac{\\partial{loss_t}}{\\partial w^2} &=& 2x^2_t\\left({\\bf w}^T{\\bf x}_t+b-y_t \\right)\\\\\n",
    "\\frac{\\partial{loss_t}}{\\partial b} &=& 2\\left({\\bf w}^T{\\bf x}_t+b-y_t \\right)\n",
    "\\end{eqnarray*}\n",
    "\n",
    "Note that the actual gradient of the loss is given by:\n",
    "$$\n",
    "\\frac{\\partial{loss}}{\\partial w^1} =\\sum_t \\frac{\\partial{loss_t}}{\\partial w^1},\\quad\n",
    "\\frac{\\partial{loss}}{\\partial w^2} =\\sum_t \\frac{\\partial{loss_t}}{\\partial w^2},\\quad\n",
    "\\frac{\\partial{loss}}{\\partial b} =\\sum_t \\frac{\\partial{loss_t}}{\\partial b}\n",
    "$$\n",
    "\n",
    "For one epoch, **(Batch) Gradient Descent** updates the weights and bias as follows:\n",
    "\\begin{eqnarray*}\n",
    "w^1_{new}&=&w^1_{old}-\\alpha\\frac{\\partial{loss}}{\\partial w^1} \\\\\n",
    "w^2_{new}&=&w^2_{old}-\\alpha\\frac{\\partial{loss}}{\\partial w^2} \\\\\n",
    "b_{new}&=&b_{old}-\\alpha\\frac{\\partial{loss}}{\\partial b},\n",
    "\\end{eqnarray*}\n",
    "\n",
    "and then we run several epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "q2eUfQws8WhA"
   },
   "outputs": [],
   "source": [
    "# randomly initialize learnable weights and bias\n",
    "w_init = random(2)\n",
    "b_init = random(1)\n",
    "\n",
    "w = w_init\n",
    "b = b_init\n",
    "print(\"initial values of the parameters:\", w, b )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "z1kUBCujB_jk",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# our model forward pass\n",
    "def forward(x):\n",
    "    return x.dot(w)+b\n",
    "\n",
    "# Loss function\n",
    "def loss(x, y):\n",
    "    y_pred = forward(x)\n",
    "    return (y_pred - y)**2 \n",
    "\n",
    "print(\"initial loss:\", np.sum([loss(x_val,y_val) for x_val, y_val in zip(x, y)]) )\n",
    "\n",
    "# compute gradient\n",
    "def gradient(x, y):  # d_loss/d_w, d_loss/d_c\n",
    "    return 2*(x.dot(w)+b - y)*x, 2 * (x.dot(w)+b - y)\n",
    " \n",
    "learning_rate = 1e-2\n",
    "# Training loop\n",
    "for epoch in range(10):\n",
    "    grad_w = np.array([0,0])\n",
    "    grad_b = np.array(0)\n",
    "    l = 0\n",
    "    for x_val, y_val in zip(x, y):\n",
    "        grad_w = np.add(grad_w,gradient(x_val, y_val)[0])\n",
    "        grad_b = np.add(grad_b,gradient(x_val, y_val)[1])\n",
    "        l += loss(x_val, y_val)\n",
    "    w = w - learning_rate * grad_w\n",
    "    b = b - learning_rate * grad_b\n",
    "    print(\"progress:\", \"epoch:\", epoch, \"loss\",l[0])\n",
    "\n",
    "# After training\n",
    "print(\"estimation of the parameters:\", w, b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "BO_DKaes8WhB"
   },
   "outputs": [],
   "source": [
    "plot_views(x, y, w, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dQz2GoXh8WhC"
   },
   "source": [
    "## Linear regression with tensors\n",
    "\n",
    "[Video timestamp](https://youtu.be/Z6H3zakmn6E?t=1650)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9KKfUKVn8WhD"
   },
   "outputs": [],
   "source": [
    "dtype = torch.FloatTensor\n",
    "# dtype = torch.cuda.FloatTensor # Uncomment this to run on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aLQeUOFi8WhF"
   },
   "outputs": [],
   "source": [
    "x_t = torch.from_numpy(x).type(dtype)\n",
    "y_t = torch.from_numpy(y).type(dtype).unsqueeze(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1MExDafv8WhG"
   },
   "source": [
    "This is an implementation of **(Batch) Gradient Descent** with tensors.\n",
    "\n",
    "Note that in the main loop, the functions loss_t and gradient_t are always called with the same inputs: they can easily be incorporated into the loop (we'll do that below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5tWn-6PC8WhH"
   },
   "outputs": [],
   "source": [
    "w_init_t = torch.from_numpy(w_init).type(dtype)\n",
    "b_init_t = torch.from_numpy(b_init).type(dtype)\n",
    "\n",
    "w_t = w_init_t.clone()\n",
    "w_t.unsqueeze_(1)\n",
    "b_t = b_init_t.clone()\n",
    "b_t.unsqueeze_(1)\n",
    "print(\"initial values of the parameters:\", w_t, b_t )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7ovfMuTVB_jr"
   },
   "outputs": [],
   "source": [
    "# our model forward pass\n",
    "def forward_t(x):\n",
    "    return x.mm(w_t)+b_t\n",
    "\n",
    "# Loss function\n",
    "def loss_t(x, y):\n",
    "    y_pred = forward_t(x)\n",
    "    return (y_pred - y).pow(2).sum()\n",
    "\n",
    "# compute gradient\n",
    "def gradient_t(x, y):  # d_loss/d_w, d_loss/d_c\n",
    "    return 2*torch.mm(torch.t(x),x.mm(w_t)+b_t - y), 2 * (x.mm(w_t)+b_t - y).sum()\n",
    "\n",
    "learning_rate = 1e-2\n",
    "for epoch in range(10):\n",
    "    l_t = loss_t(x_t,y_t)\n",
    "    grad_w, grad_b = gradient_t(x_t,y_t)\n",
    "    w_t = w_t-learning_rate*grad_w\n",
    "    b_t = b_t-learning_rate*grad_b\n",
    "    print(\"progress:\", \"epoch:\", epoch, \"loss\",l_t)\n",
    "\n",
    "# After training\n",
    "print(\"estimation of the parameters:\", w_t, b_t )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bTrqi-ux8WhJ"
   },
   "source": [
    "## Linear regression with Autograd\n",
    "\n",
    "[Video timestamp](https://youtu.be/Z6H3zakmn6E?t=1890)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LTH6VOMz8WhJ"
   },
   "outputs": [],
   "source": [
    "# Setting requires_grad=True indicates that we want to compute gradients with\n",
    "# respect to these Tensors during the backward pass.\n",
    "w_v = w_init_t.clone().unsqueeze(1)\n",
    "w_v.requires_grad_(True)\n",
    "b_v = b_init_t.clone().unsqueeze(1)\n",
    "b_v.requires_grad_(True)\n",
    "print(\"initial values of the parameters:\", w_v.data, b_v.data )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oHgm3N8Y8WhK"
   },
   "source": [
    "An implementation of **(Batch) Gradient Descent** without computing explicitly the gradient and using autograd instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "B4sdOF0e8WhL"
   },
   "outputs": [],
   "source": [
    "for epoch in range(10):\n",
    "    y_pred = x_t.mm(w_v)+b_v\n",
    "    loss = (y_pred - y_t).pow(2).sum()\n",
    "    \n",
    "    # Use autograd to compute the backward pass. This call will compute the\n",
    "    # gradient of loss with respect to all Variables with requires_grad=True.\n",
    "    # After this call w.grad and b.grad will be tensors holding the gradient\n",
    "    # of the loss with respect to w and b respectively.\n",
    "    loss.backward()\n",
    "    \n",
    "    # Update weights using gradient descent. For this step we just want to mutate\n",
    "    # the values of w_v and b_v in-place; we don't want to build up a computational\n",
    "    # graph for the update steps, so we use the torch.no_grad() context manager\n",
    "    # to prevent PyTorch from building a computational graph for the updates\n",
    "    with torch.no_grad():\n",
    "        w_v -= learning_rate * w_v.grad\n",
    "        b_v -= learning_rate * b_v.grad\n",
    "    \n",
    "    # Manually zero the gradients after updating weights\n",
    "    # otherwise gradients will be accumulated after each .backward()\n",
    "    w_v.grad.zero_()\n",
    "    b_v.grad.zero_()\n",
    "    \n",
    "    print(\"progress:\", \"epoch:\", epoch, \"loss\",loss.data.item())\n",
    "\n",
    "# After training\n",
    "print(\"estimation of the parameters:\", w_v.data, b_v.data.t() )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "W720TY4R8WhN"
   },
   "source": [
    "## Linear regression with neural network\n",
    "\n",
    "[Video timestamp](https://youtu.be/Z6H3zakmn6E?t=2075)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6VgMVkn48WhN"
   },
   "source": [
    "An implementation of **(Batch) Gradient Descent** using the nn package. Here we have a super simple model with only one layer and no activation function!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "plEHj42d8WhO"
   },
   "outputs": [],
   "source": [
    "# Use the nn package to define our model as a sequence of layers. nn.Sequential\n",
    "# is a Module which contains other Modules, and applies them in sequence to\n",
    "# produce its output. Each Linear Module computes output from input using a\n",
    "# linear function, and holds internal Variables for its weight and bias.\n",
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 1),\n",
    ")\n",
    "\n",
    "for m in model.children():\n",
    "    m.weight.data = w_init_t.clone().unsqueeze(0)\n",
    "    m.bias.data = b_init_t.clone()\n",
    "\n",
    "# The nn package also contains definitions of popular loss functions; in this\n",
    "# case we will use Mean Squared Error (MSE) as our loss function.\n",
    "loss_fn = torch.nn.MSELoss(reduction='sum')\n",
    "\n",
    "# switch to train mode\n",
    "model.train()\n",
    "\n",
    "for epoch in range(10):\n",
    "    # Forward pass: compute predicted y by passing x to the model. Module objects\n",
    "    # override the __call__ operator so you can call them like functions. When\n",
    "    # doing so you pass a Variable of input data to the Module and it produces\n",
    "    # a Variable of output data.\n",
    "    y_pred = model(x_t)\n",
    "  \n",
    "    # Note this operation is equivalent to: pred = model.forward(x_v)\n",
    "\n",
    "    # Compute and print loss. We pass Variables containing the predicted and true\n",
    "    # values of y, and the loss function returns a Variable containing the\n",
    "    # loss.\n",
    "    loss = loss_fn(y_pred, y_t)\n",
    "\n",
    "    # Zero the gradients before running the backward pass.\n",
    "    model.zero_grad()\n",
    "\n",
    "    # Backward pass: compute gradient of the loss with respect to all the learnable\n",
    "    # parameters of the model. Internally, the parameters of each Module are stored\n",
    "    # in Variables with requires_grad=True, so this call will compute gradients for\n",
    "    # all learnable parameters in the model.\n",
    "    loss.backward()\n",
    "\n",
    "    # Update the weights using gradient descent. Each parameter is a Tensor, so\n",
    "    # we can access its data and gradients like we did before.\n",
    "    with torch.no_grad():\n",
    "        for param in model.parameters():\n",
    "            param.data -= learning_rate * param.grad\n",
    "        \n",
    "    print(\"progress:\", \"epoch:\", epoch, \"loss\",loss.data.item())\n",
    "\n",
    "# After training\n",
    "print(\"estimation of the parameters:\")\n",
    "for param in model.parameters():\n",
    "    print(param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gus-YF9p8WhP"
   },
   "source": [
    "Last step, we use directly the optim package to update the weights and bias.\n",
    "\n",
    "[Video timestamp](https://youtu.be/Z6H3zakmn6E?t=2390)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2VbF5gYV8WhP"
   },
   "outputs": [],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 1),\n",
    ")\n",
    "\n",
    "for m in model.children():\n",
    "    m.weight.data = w_init_t.clone().unsqueeze(0)\n",
    "    m.bias.data = b_init_t.clone()\n",
    "\n",
    "loss_fn = torch.nn.MSELoss(reduction='sum')\n",
    "\n",
    "model.train()\n",
    "\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n",
    "\n",
    "\n",
    "for epoch in range(10):\n",
    "    y_pred = model(x_t)\n",
    "    loss = loss_fn(y_pred, y_t)\n",
    "    print(\"progress:\", \"epoch:\", epoch, \"loss\",loss.item())\n",
    "    # Zero gradients, perform a backward pass, and update the weights.\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    \n",
    "# After training\n",
    "print(\"estimation of the parameters:\")\n",
    "for param in model.parameters():\n",
    "    print(param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vFxayCILB_j2"
   },
   "source": [
    "## Remark\n",
    "\n",
    "This problem can be solved in 3 lines of code!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6q9sz3M8B_j2"
   },
   "outputs": [],
   "source": [
    "xb_t = torch.cat((x_t,torch.ones(30).unsqueeze(1)),1)\n",
    "# for old version of pytorch\n",
    "#sol, _ =torch.lstsq(y_t,xb_t)\n",
    "#sol[:3]\n",
    "# for pytorch 1.9 and newer\n",
    "sol = torch.linalg.lstsq(xb_t,y_t)\n",
    "sol.solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IY_I9v3o8WhQ"
   },
   "source": [
    "## Exercise: Play with the code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "It_tRuXs8WhQ"
   },
   "source": [
    "Change the number of samples from 30 to 300. What happens? How to correct it?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JRk1zgSv8WhR"
   },
   "outputs": [],
   "source": [
    "x = random((300,2))\n",
    "y = np.dot(x, [2., -3.]) + 1.\n",
    "x_t = torch.from_numpy(x).type(dtype)\n",
    "y_t = torch.from_numpy(y).type(dtype).unsqueeze(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "a-xOCI6z8WhS"
   },
   "outputs": [],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 1),\n",
    ")\n",
    "\n",
    "for m in model.children():\n",
    "    m.weight.data = w_init_t.clone().unsqueeze(0)\n",
    "    m.bias.data = b_init_t.clone()\n",
    "\n",
    "loss_fn = torch.nn.MSELoss(reduction = 'sum')\n",
    "\n",
    "model.train()\n",
    "\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n",
    "\n",
    "\n",
    "for epoch in range(10):\n",
    "    y_pred = model(x_t)\n",
    "    loss = loss_fn(y_pred, y_t)\n",
    "    print(\"progress:\", \"epoch:\", epoch, \"loss\",loss.item())\n",
    "    # Zero gradients, perform a backward pass, and update the weights.\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    \n",
    "# After training\n",
    "print(\"estimation of the parameters:\")\n",
    "for param in model.parameters():\n",
    "    print(param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CAiS-TfqDOMU"
   },
   "source": [
    "[![Dataflowr](https://raw.githubusercontent.com/dataflowr/website/master/_assets/dataflowr_logo.png)](https://dataflowr.github.io/website/)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "include_colab_link": true,
   "name": "02_basics.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
